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A New Method 

Goals 

Develop a behavioral measure of decision state that: 

- tracks the development of decisions over time 

- allows observers to indicate uncertainty with graded responses 
Track influences of top-down and bottom-up information overtime 

Task 

Observers determined which of two patches of dots had more dots. 
Observers were asked to indicate their confidence that more dots 
were on the right or left by positioning a small bar with a joystick. 
Observers were asked to respond as quickly as possible. 

3:1 odds 3:1 odds 



Scoring 

Accuracy 

Observers received points as a function of their distance from the 
correct response. 

The function gave the highest expected value when observers 
accurately reported the probability they were correct. 




Speed 

Trials ended at a random point in 
time, not controlled by the 
observer. 

Observers received a score based 
on the bar position at the end 
of the trial. 

Thus observers had to move the bar 
as quickly as possible when new 
information became available. 

Trial Duration (sec) 

Design 

We manipulated bottom-up discriminability across trials. One patch 
always had 50 dots, the other had 5 1 , 52, 54, 58 or 66. 

We manipulated top-down expectation across blocks. One patch was 
three times as likely to have more dots as the other. 

Observers saw 1 0 blocks of 82 trials. The first 3 blocks of the session and 
the first 2 trials of every block were discarded. The remaining 80 trials/block 
contained 15 trials of each discriminability level on the expected side and 5 on 
the unexpected side. 



How well does 
the method work? 

Overall Performance 

Performance reached asymptote after about 3 blocks (246 trials). Some observers performed 
much better than others. Almost half of the observers scored fewer points than could have been 
scored by leaving the bar indicating 3:1 odds more dots will be on the expected side (2800 pts). 



Reporting of Confidence 

No observers were close to optimal in the absence of perceptual information (e.g., before 
stimulus onset). With perceptual information, all observers showed some degree of graded 
responding, with the best observers reporting their confidence fairly accurately. 


Worst 5 Os Middle 5 Os Best 5 Os 



Bar Position Bar Position Bar Position 


Worst 5 Os 


Middle 5 Os 


Best 5 Os 




Bar Position Bar Position Bar Position 


Worst 5 Os 


CO 1 

Eg 



P 


§ 

I 




Middle 5 Os 



Best 5 Os 





What can we learn 
using this method? 

The data presented below is colored ^ 33 
according to this convention. 52 vs 50 dots 



Looking at Individual Trials 


Unlike traditional methods, we can get some idea of observer's state of mind on individual trials. 



Examining patterns: How often do 
observers change their mind? 

If you find a particular data pattern of interest you can pull out the corresponding trials and 
characterize them. Here we look at the first place the observer set the bar plotted against the 
final bar position to find trials on which the observer changed their mind. 
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Expected and unexpected trials can 
be looked at independently. 

One can separate bias from sensitivity more easily than with signal detection. 


Mean, across all Os, for each difficulty 
level and point in time 
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Mean, across all Os, for 54 vs 50 dot 
condition at each point in time 




move the bar toward 
the expected side. 


Conclusions 


About half the observers 
reach asymptote more slowly 
when there were more dots 
on the unexpected side. 


We have created a paradigm that allows us to capture the evolution of observer decision state 
over time. While not all observers accurately estimated the probability that they were correct, we 
can, for all observers, track the time course of their decision process. Doing so gives us an 
ability to examine single trials at a level of detail that is not captured with traditional 
psychophysical techniques. 
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